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(54) Data multiplexing apparatus and method 



(57) In a data multiplexer for multiplexing and out- 
putting data streams of plural channels, multiplexing ef- 
ficiency can be improved by effectively using a data 
transmission rate of a transmission system. Invalid data 



assigned to an inputted data stream is detected and ex- 
tracted and moreover, a data stream from which the 
invalid data is extracted is multiplexed and outputted, so 
that multiplexing efficiency for valid data can be im- 
proved. 
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Description 

[0001] The present invention relates to a data multi- 
plexer and a multiplexing method and more particularly, 
is suitably applied, for example, to a data multiplexer of 
a digital broadcasting system. 

[0002] Various compression-encoding methods are 
recently proposed as methods of reducing the informa- 
tion quantity of pictures and sounds and Moving Picture 
Experts Group Phase 2 (MPEG2) is the typical one of 
the compression-encoding methods. 
[0003] The MPEG2 method is standardized (e.g. Rec- 
ommendation H.222.0) by an organization such as In- 
ternational Telecommunication Union-Telecommunica- 
tion Standardization Sector (ITU-T) and specified in or- 
der to compression-encode, multiplex and transmit pic- 
ture data and sound data. 

[0004] Actually, the MPEG2 method specifies a data 
format referred to as a program stream (hereafter re- 
ferred to as a PS data format) and a data format referred 
to as a transport stream (hereafter, referred to as a TS 
data format) as data formats for multiplexing compres- 
sion-encoded picture data and sound data. 
[0005] The PS data format is used to accumulate mul- 
tiplexed picture data and sound data in a predetermined 
digital accumulation medium, and the TS data format is 
used to transmit multiplexed picture data and sound da- 
ta. Moreover, streams having the PS data format and 
TS data format can be mutually converted with a varia- 
ble-length Packetized Elementary Stream (PES) packet 
as a basic element. 

[0006] In the case of the PS data format, for example, 
it is specified that compression-encoded picture data 
and sound data are respectively formed into a PES 
packet every predetermined unit (e.g. every picture), to 
form a PS packet string (so-called, a program stream) 
by time-dividing and multiplexing each of the resultant 
PES packets. 

[0007] Fig. 1 shows a transmitter 100 of a digital 
broadcasting system. The transmitter 100 supplies pic- 
ture data D1 A and D1 B corresponding to respective tel- 
evision broadcast programs to corresponding encoders 
1 01 A and 101 B from data output sections (not illustrat- 
ed) corresponding to plural channels, compression-en- 
codes the picture data D1 A and D1 B in accordance with 
the MPEG2 method in the corresponding encoders 
101 A and 101 B, and successively forms encoded data 
thus obtained into a PES packet every picture, in order 
to form and supply encoded data streams D31 A and 
D31 B to a multiplexer 1 02. 

[0008] To multiplex the encoded data streams D31 A 
and D31B supplied from the respective encoders 101 A 
and 101B, the multiplexer 102 stores and temporarily 
accumulates plural PES packets included in the encod- 
ed data streams D31 A and D31 B in corresponding buff- 
ers 1 03A and 1 03B in arriving order and then, reads out 
and transmits the PES packets from the buffers to a mul- 
tiplexing section 104 in accordance with the first-in first- 



out method. 

[0009] In this case, the buffers 103A and 103B re- 
spectively supply the amount of data accumulated in the 
PES packets to a multiplexing control section 1 05 as da- 
5 ta accumulation signals S2A and S2B and thereby, the 
multiplexing control section 1 05 respectively detects the 
amount of data accumulated in the buffers 103A and 
103B, based on the data accumulation signals S2A and 
S2B. 

io [0010] The multiplexing control section 105 transmits 
buffer output control signals S3A and S3B to the respec- 
tive buffers 103A and 103B based on the detection re- 
sult before the amount of data exceed the maximum da- 
ta accumulation capacities of the buffers 103A and 

*5 103B. Thereby, the section 105 can read out the PES 
packets from the buffers 103A and 103B in accordance 
with the first-in first-out method and thus, it is possible 
to prevent the buffers 1 03A and 1 03B from being broken 
down due to excessive data accumulation. 

20 [0011] Then, the multiplexing section 104 receives the 
encoded data streams D31 A and D31 B from the respec- 
tive buffers 103A and 103B and moreover, receives a 
multiplexing control signal S4 from the multiplexing con- 
trol section 105 and then, successively forms plural PES 

25 packets included in the respective encoded data 
streams D31 A and D31 B into a packet (hereafter, re- 
ferred to as transport stream "TS" packet) every prede- 
termined unit (e.g. every 188-byte data) and thereafter, 
forms a transport stream (TS stream) comprising one 

30 TS packet string D32 by time-dividing and multiplexing 
the resultant TS packets. 

[0012] Then, the TS stream D32 is modulated by a 
not-illustrated modulator in accordance with a predeter- 
mined method and the resultant transmission signal is 
35 transmitted toward a broadcasting satellite (not illustrat- 
ed) through an antenna. In this way, the transmitter 100 
can broadcast television programs for plural channels 
through the broadcasting satellite. 
[001 3] In the case of the transmitter 1 00, the encoders 
40 1 01 A and 1 01 B supply the respective compression-en- 
coded picture data (encoded data streams D31 A and 
D31B) to a multiplexer at a fixed bit rate. In this case, 
when patterns of input picture data D1 A and D1B are 
flat, the amount of encoded data successively generat- 
es ed by the encoders 101 A and 101B are reduced due to 
the fact that the amount of picture information of the pic- 
ture data D1 A and D1 B are extremely small. 
[0014] Therefore, when the encoders 101 A and 101 B 
form successively generated encoded data into a PES 
50 packet every picture, they keep the transmission rates 
of the encoded data streams D31 A and D31 B compris- 
ing the PES packet strings constant by adding dummy 
data originally unnecessary for encoding (hereafter, re- 
ferred to as invalid data) to valid data in bytes. 
55 [0015] The transmission rates ol significant informa- 
tion in the encoded data streams D31 A and D31 B have 
a problem of being reduced by a value equivalent to add- 
ed invalid data because the encoders 101 A and 101 B 
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respectively output encoded data streams D31A and 
D31 B to which invalid data is added. 
[001 6] I n view of the foregoing, an object of this inven- 
tion is to provide a data multiplexer in which multiplexing 
efficiency can be improved by effectively using the data 
transmission rate of a transmission system. 
[0017] The foregoing object and other objects of the 
invention have been achieved by the provision of a data 
multiplexer which detects and extracts invalid data as- 
signed to an inputted data stream and multiplexes and 
outputs the data stream from which the invalid data is 
extracted. Thereby, it is possible to improve multiplexing 
efficiency for invalid data. 

[0018] Moreover, by detecting the amount of extract- 
ed invalid data and multiplexing a predetermined data 
stream onto the data stream from which invalid data is 
extracted in accordance with the amount of detected 
invalid data, it is possible to improve multiplexing effi- 
ciency by a value equivalent to the multiplexed prede- 
termined data stream instead of the invalid data. 
[0019] Embodiments of the invention will now be de- 
scribed, by way of example, with reference to the ac- 
companying drawings in which: 

Fig. 1 is a block diagram showing a transmitter; 
Fig. 2 is a block diagram showing a first embodi- 
ment of a multiplexer according to the present in- 
vention; 

Fig. 3 is a block diagram showing the structure of 
an encoder in the first embodiment; 
Figs. 4A and 4B are schematic diagrams showing 
structures of a PES packet; 
Figs. 5A to 5H are schematic diagrams explaining 
the process of extracting invalid data; 
Fig. 6 is a block diagram showing a second embod- 
iment of a multiplexer according to the present in- 
vention; 

Fig. 7 is a block diagram showing the structure of 
an encoder in the second embodiment; and 
Figs. 8A to 8H are schematic diagrams explaining 
the processing of extracting invalid data according 
to other embodiments. 

[0020] Preferred embodiments of this invention will be 
described with reference to the accompanying draw- 
ings: 

(1) First Embodiment 

[0021] In Fig. 2 where the same reference numerals 
are applied to parts corresponding to Fig. 1, numeral 
120 denotes a transmitter of a digital broadcasting sys- 
tem according to the first embodiment, to which the 
present invention is applied as a whole. In the transmit- 
ter 120, picture data D1 A and D1B constituting televi- 
sion broadcast programs are respectively supplied to 
encoders 101 A and 1 01 B from data output sections (not 
illustrated) corresponding to plural of channels. 



[0022] In this case, the encoders 1 01 A and 101 B have 
the same structure. The encoder 1 01 A inputs the picture 
data D1 A to a preprocessing section 11 A as shown in 
Fig. 3. The preprocessing section 11 A designates as 
s which type each frame picture of successively inputted 
picture data D1 A should be processed among three pic- 
ture types of l-picture, P-picture, and B-picture and then, 
rearranges the frame pictures in order of encoding in 
accordance with the picture type of the frame pictures, 
io divides the frame pictures into macroblocks respectively 
constituted with a luminance signal of 16 pixels x 16 
lines and a color difference signal corresponding to the 
luminance signal, and supplies the macroblocks to an 
arithmetic circuit 1 2A and a movement vector detecting 
section 23A as macroblock data D2A. 
[0023] The movement vector detecting section 23A 
computes the movement vector of each macroblock of 
the macroblock data D2A in accordance with the mac- 
roblock data D2A and reference picture data D14A 
stored in a frame memory 21 A and transmits the com- 
puted movement vector to a movement compensating 
section 22A as movement vector data D18A. 
[0024] The arithmetic circuit 12A applies any move- 
ment compensation of the intramode ; forward estima- 
tion mode, backward estimation mode, and bidirectional 
estimation mode to the macroblock data D2A supplied 
from the preprocessing section 11 A in accordance with 
the picture type of each macroblock of the macroblock 
data D2A. In this case, the intramode is a method of us- 
ing a frame picture to be encoded directly as transmis- 
sion data, and the forward estimation mode is a method 
of using the estimation residue between a frame picture 
to be encoded and a past reference picture as transmis- 
sion data. The backward estimation mode is a method 
of using the estimation residue between a frame picture 
to be encoded and a future reference picture as trans- 
mission data, and the bidirectional estimation mode is 
a method of using the estimation residue between a 
frame picture to be encoded and the average value of 
a past reference picture and a future reference picture 
as transmission data. 

[0025] First, the case is described in which the mac- 
roblock data D2A is an l-picture. In this case, the mac- 
roblock data D2A is processed in the intramode. That 
is, the arithmetic circuit 12A directly transmits the mac- 
roblock of the macroblock data D2A to a discrete cosine 
transform (DCT) section 1 3A as arithmetic data D3A. 
The DCT section 1 3A applies the DCT processing to the 
arithmetic data D3A to transform the "data D3A into a 
DCT coefficient and transmits the DCT coefficient to a 
quantizing section 14A as DCT coefficient data D4A. 
The quantizing section 14A applies quantization 
processing to the DCT coefficient data D4A and trans- 
mits it to a VLC section 15A and a reverse -quantizing 
section 18A as quantized DCT coefficient data D5A. in 
this case, the quantizing section 14A adjusts a quanti- 
zation step size in the quantization processing in ac- 
cordance with a quantization control value D21 A sup- 
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plied from a control section 142A in order to control the 
number of codes to be generated. 
[0026] The quantized DCT coefficient data D5A trans- 
mitted to the reverse-quantizing section 1 BA undergoes 
the reverse-quantization processing to be transmitted to 
a reverse-DCT section 19A as DCT coefficient data 
D11 A. Then, the DCT coefficient data D11 A undergoes 
the reverse-DCT processing in the reverse-DCT section 
19A to be transmitted to an arithmetic circuit 20A as 
arithmetic data D12A and stored in the frame memory 
21 A as reference picture data D13A. 
[0027] Then, the case is described in which the mac- 
roblock data D2A is a P-picture. In this case : the arith- 
metic circuit 12A applies the movement compensation 
processing based on either the intramode or forward es- 
timation mode, to the macroblock data D2A. 
[0028] In the case where the intramode is used as the 
estimation mode, the arithmetic circuit 12A directly 
transmits the macroblock of the macroblock data D2A 
to the DCT section 13A as the arithmetic signal D3A as 
the same as the case of the above l-picture. 
[0029] In the case where the forward estimation mode 
is used as an estimation mode, however, the arithmetic 
circuit 12A applies a subtraction processing to the mac- 
roblock data D2A by using forward-estimation picture 
data D17A supplied from the movement compensating 
section 22A. 

[0030] The forward-estimation picture data D17A is 
computed by movement-compensating the reference 
picture data D13A, which is stored in the frame memory 
21 A, in accordance with the movement vector data 
D18A. That is, the movement compensating section 22 
reads out the reference picture data D1 3A in the forward 
estimation mode by shifting the read address of the 
frame memory 21 A in accordance with the movement 
vector data D18A and supplies the data D13A to the 
arithmetic circuits 12A and 20A as the forward-estima- 
tion picture data D17A. The arithmetic circuit 12A sub- 
tracts the forward-estimation pictu re data D1 7A from the 
macroblock data D2A to obtain difference data as esti- 
mation residue and transmits the difference data to the 
DCT section 13A as the arithmetic data D3A. 
[0031] Moreover, the forward-estimation picture data 
D17A is supplied to the arithmetic circuit 20A from the 
movement compensating section 22A. The arithmetic 
circuit 20A locally regenerates the reference picture da- 
ta D13A (P-picture) by adding the forward-estimation 
picture data D17A to the arithmetic data D12A and 
stores the data D1 3A in the frame memory 21 A. 
[0032] Then, the case is described in which the mac- 
roblock data D2A being the B-picture is supplied to the 
arithmetic circuit 12A from the preprocessing section 
11 A. In this case, the" arithmetic circuit 12A applies any 
movement compensation processing of the intramode, 
forward estimation mode, backward estimation mode, 
and bidirectional estimation mode to the macroblock da- 
ta D2A. 

[0033] In the case where the intramode or forward 



mode is used as the estimation mode, macroblock data 
S2 undergoes the same processing as the case of the 
above P-picture. However, because the B-picture is not 
used as other estimation reference picture, the refer- 
s ence picture data D1 3A is not stored in the frame mem- 
ory 21 A. 

[0034] On the other hand, in the case where the back- 
ward estimation mode is used as the estimation mode, 
the arithmetic circuit 12A applies the subtraction 
10 processing to the macroblock data D2A by using back- 
ward-estimation picture data D16A supplied from the 
movement compensating section 22A 
[0035] The backward-estimation picture data D1 6 A is 
computed by movement-compensating the reference 
is picture data D13A stored in the frame memory 21 A in 
accordance with the movement vector data D1BA. That 
is, the movement compensating section 22A reads out 
the reference picture data D13A in the backward esti- 
mation mode by shifting the read address of the frame 
memory 21 A in accordance with the movement vector 
data D18A, and supplies the data D1 3Ato the arithmetic 
circuits 12Aand20Aasthe backward-estimation picture 
data D16A. The arithmetic circuit 12A subtracts the 
backward-estimation picture data D16A from the mac- 
roblock data D2A to obtain difference data as estimation 
residue and transmits the difference data to the DCT 
section 1 3A as the arithmetic data D3A. 
[0036] Moreover, the backward-estimation picture da- 
ta D1 6A is supplied to the arithmetic circuit 20A from the 
movement compensating section 22A and the arithme- 
tic circuit 20A locally regenerates the reference picture 
data D13A (B-picture) by adding the backward-estima- 
tion picture data D16A to the arithmetic data D12A. 
However, the reference picture data D1 3A is not stored 
in the frame memory 21 A because the B-picture is not 
used as an estimation reference picture. 
[0037] In the case where the bidirectional mode is 
used as an estimation mode, the arithmetic circuit 12A 
subtracts the average value of the forward-estimation 
picture data D17A and backward-estimation picture da- 
ta D16A, which are supplied from the movement com- 
pensating section 22A, from the macroblock data D2A 
to obtain difference data as estimation residue and 
transmits the difference data to the DCT section 1 3A as 
the arithmetic data D3A. 

[0038] Moreover, the forward-estimation picture data 
D17A and the backward-estimation picture data D16A 
are supplied to the arithmetic circuit 20A from the move- 
ment compensating section 22A and the arithmetic cir- 
cuit 20 A generates the reference picture data Dl 3A (B- 
picture) by adding the average value of the forward-es- 
timation picture data D17A and the backward-estima- 
tion picture data D16A to the arithmetic data D12A. 
However, the reference picture data D1 3A is not stored 
in the frame memory 21 A because the B-picture is not 
used as another estimation reference picture 
[0039] Thus, the picture data D1 A inputted to the en- 
coder 101 A undergoes the movement compensation 
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processing, DCT processing, and quantization process- 
ing to be supplied to the VLC section 15A as the quan- 
tized DCT coefficient data D5A. 
[0040] The VLC section 15A applies variable-length 
encoding processing based on a predetermined trans- 
form table to the quantized DCT coefficient data D5A 
and transmits the resultant data to a buffer section 141 A 
as variable-length encoded data D6A. 
[0041] The control section 142A always monitors the 
accumulation state of the variable-length encoded data 
D6A in the buffer section 141 A and obtains the above 
accumulation state as occupied amount information 
D23A. Moreover, the control section 142A generates 
such the amount of data that the buffer section 141 A 
does not overflow or under-f low, in accordance with the 
amount of data occupying the buffer 141 A, by generat- 
ing the quantization control value D21 A in accordance 
with the occupied amount information D23A, transmit- 
ting the value D21 A to the quantizing section 14A, and 
adjusting the quantization step size in the quantization 
processing. 

[0042] The buffer section 1 41 A transmits the accumu- 
lated variable-length encoded data D6A to a mu Itiplexer 
121 (Fig. 2) as an output data stream DBA constituting 
a PES every picture at a constant rate in accordance 
with a data transmission control signal D22A supplied 
from the control section 142A. In this case, the buffer 
section 141 A of the encoder IOIA can keep the rate of 
the output data stream D8A outputted from the buffer 
section 141 A constant even if the amount of data gen- 
erated in the quantizing section 14A is extremely re- 
duced and thereby the amount of variable-length encod- 
ed data D6A accumulated in the buffer 141 A is reduced, 
because dummy data is added to an optional position 
of the output data stream D8 A at an invalid-data adding 
section (not illustrated) provided at the output stage of 
the buffer section 141 A in accordance with the amount 
of encoded data accumulated in the buffer section 141 A 
when the amount of encoded data is extremely small. 
[0043] That is, as shown in Fig. 4A, the PES packet 
constituting the output data stream DBA outputted from 
the encoder 101 A is composed of a header part and a 
data part, and a packet start code prefix, PES header 
option, and other various packet control data are stored 
in the header part and encoded data is stored in the data 
part. 

[0044] In this case where the pattern of the picture da- 
ta D1 A which is inputted to the encoder 1 01 A is flat and 
the picture information based on the picture data D1 A 
is extremely small, the encoder IOIA forms an invalid da- 
ta part for dummy data referred to as stuffing-byte data 
between the header part and the invalid-data part in the 
data part of the PES packet (after the header part) when 
forming encoded data into a PES packet every picture 
and outputting the PES packet from the buffer 141 A, as 
shown in Fig. 4B. In this case, the amount of data is 
assigned as the dummy data in bytes so as to keep the 
transmission rate of the output data stream DBA output- 



ted from the encoder IOIA at a preset rate. 
[0045] In this connection, the boundary between the 
invalid data part, to which dummy data is assigned, and 
the valid data part, to which encoded picture data (valid 

s data) is assigned, can be detected by the header part 
of the picture data added to the head of the valid data. 
[0046] Thus, the output data stream D8A comprising 
a PES packet string is outputted to the multiplexer 121 
from the encoder 101 A, keeping the transmission rate 

io constant. 

[0047] Though the above mention relates to the first 
encoder 101A, other encoders 101Band 124alsohave 
the same structure. 

[0048] To multiplex the output data streams D8A and 
is D8B supplied from the respective encoders 101 A and 
101 B, the multiplexer 121 stores and temporarily accu- 
mulates a plurality of PES packets included in the output 
data streams D8A and D8B in their corresponding buff- 
ers 1 03A and 1 03B in arriving order, and the data accu- 

20 mulated in the buffers 103A and 103B are read out in 
accordance with the first-in first-out method to be trans- 
mitted to an invalid data extracting section 122. 
[0049] In this case, the buffers 103A and 103B re- 
spectively supply the amount of data of the accumulated 

25 PES packets to the multiplexing control section 123 as 
the data accumulation signals S2A and S2B and there- 
by, the multiplexing control section 123 detects the 
amount of data which occupy the buffers 1 03A and 1 03B 
in accordance with the respective data accumulation 

30 signals S2A and S2B. 

[0050] The multiplexing control section 123 can read 
out PES packets from the buffers 103Aand 103B in ac- 
cordance with the first-in first-out method by transmitting 
the buffer output control signals S3A and S3B to the 

35 buffers 1 03A and 1 03B respectively in accordance with 
the above detection result before the buffers 103A and 
103B exceed their maximum data accumulation capac- 
ity and thus, it is possible to prevent the buffers 103A 
and 103B from being broken down due to excessive da- 

40 ta accumulation. 

[0051] Then, the invalid data extracting section 122 
successively extracts dummy data from the respective 
PES packets of the output data streams D8A and DBB, 
which are successively read out from the buffers 103 A 

4S and 103B, and thereafter, respectively transmits the 
dummy data to a multiplexing section 127 as valid data 
streams D8A' and D8B' and counts the number of ex- 
tracted dummy data and transmits the counted result to 
the multiplexing control section 123 as an invalid-data- 

50 amount-information signal S1 0. 

[0052] Specifically, as shown in Figs. 5A and 5B, in 
the case where invalid data parts ND A2 and ND M are 
formed in, for example, respective PES packets P A2 and 
P A 4 among PES packet strings P A1 , P^, P^, P Ml ... 

55 forming the output data stream D8A outputted from the 
encoder 101 A and invalid data parts ND B1 , ND B2 , and 
ND B3 are formed in, for example, respective PES pack- 
ets P B1 , P B2 , and P B3 among PES packet strings P B1 , 
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P B2« p 83. P B4' — forming the output data stream D8B 
outputted from the encoder 101B, the invalid data ex- 
tracting section 122 successively detects the header 
parts H A1 , H^, H A3 , and H A4 and H Bl , H B2 , H B3 , and 
Hb4 oi a coming PES packet string on the respective 
output data streams D8A and D8B in order to judge 
whether the data detected after each header part is 
dummy data. In this case, since dummy data is data not 
used as valid data (e.g. "00" or "FF"), the invalid data 
extracting section 122 can easily discriminate the dum- 
my data (ND^ and ND A4 , and ND B1 , ND B2 , and ND B3 ). 
[0053] Moreover, because the header part of picture 
data showing the beginning of valid data is assigned to 
the head of valid data the invalid data extracting section 
122 can discriminate the start of the valid data by de- 
tecting the header part of the picture data. 
[0054] Thus, as shown in Figs. 5C, 5D, and 5E, the 
invalid data extracting section 122 does not allow the 
dummy data ND^, ND A4 , ND B1 , ND B2 , or ND^ in the 
respective output data streams DBA and D8B com- 
posed of PES packet strings to pass but it outputs only 
valid data and header parts other than the dummy data. 
[0055] As a result, valid data streams DBA' and DBB' 
outputted from the invalid data extracting section 122 
are constituted using the PES packets and P^', 
and P B1 \ P B2 ' 5 and P B3 ' each of whose data length be- 
came short because the dummy data ND^ and ND A4 , 
and ND B1 , ND B2 , and ND B3 are extracted, as shown in 
Figs. 5G and 5H. 

[0056] Moreover, as shown in Fig. 5E, by counting the 
dummy data ND A2 and ND^, and ND B1 , ND B2 , and 
ND B3 for a predetermined time whenever extracting the 
dummy data from the output data streams D8A and 
D8B, the invalid data extracting section 122 transmits 
the amount of counted dummy data to the multiplexing 
control section 123 as the invalid-data-amount-informa- 
tion signal S10 as shown in Fig. 5F By sequentially 
counting the amount of dummy data shown by the 
invalid-data-amount-information signal S10 supplied 
from the invalid data extracting section 122, the multi- 
plexing control section 123 sequentially detects the total 
amount of the dummy data extracted from the output 
data streams DBA and DBB outputted from the encoders 
101 A and 101B. 

[0057] Thus, the valid data streams D8A' and D8B' 
outputted from the invalid data extracting section 122 
lose its total amount of data because the dummy data 
ND^ and ND M , and ND B1 , ND B2 , and ND B3 are extract- 
ed and under the above state, the data streams D8A' 
and D8B* are outputted to the multiplexing section 127 
and recognized using the invalid-data-quantity-informa- 
tion signal S10 by the multiplexing control section 123. 
[0058] The multiplexing control section 1 23 transmits 
a multiplexing control signal S1 6 to the multiplexing sec- 
tion 127 in accordance with the multiplexing schedule 
obtained depending on the amount of data which occu- 
py in the buffers 103A and 103B. The multiplexing sec- 
tion 127 generates a transport stream (TS) packet hav- 



ing a fixed data length in accordance with the valid data 
streams D8A' and D8B' respectively having the PES 
structure, multiplexes the TS packet in order to generate 
a multiplexed stream (transport stream) D15, and out- 

5 puts the multiplexed stream to a transmission system 
(not illustrated) including a transmission antenna. 
[0059] In this connection, it is necessary to correct 
time information for the valid data streams DBA' and 
DBB' because dummy data are extracted at the invalid 

10 data extracting section 122. Therefore, the multiplexing 
control section 123 adds new time information by trans- 
mitting the multiplexing control signal S16 to the multi- 
plexing section 127. 

[0060] In addition to the above structure, in the trans- 
it mitter 120, service data D1X such as personal-compu- 
ter communication data and information service data is 
supplied to the encoder 124 from a not-illustrated data 
output section when required. The encoder 1 24 succes- 
sively forms the generated encoded data into PES pack- 
20 ets in order for each predetermined data unit in accord- 
ance with the aforementioned compression-encoding of 
the MPEG2 method, in order to form and supply an out- 
put data stream D8X to a corresponding buffer 125 in 
the multiplexer 121. In this connection, the service data 
2S D1 X is data which can be accumulated and is unneces- 
sary to transmit in real time but it can be used according 
to necessity after being accumulated at the receiving 
side. 

[0061] Plural PES packets included in the output data 
30 stream D8X are successively stored and accumulated 
in the buffer 125 in arriving order. In this case, the buffer 
125 supplies the amount of data of the accumulated 
PES packets to the multiplexing control section 123 as 
a data accumulation signal S12 and thereby, the multi- 
35 plexing control section 123 can recognize the amount 
of data accumulated in the buffer 1 25 based on the data 
accumulation signal S12. 

[0062] In this case, the multiplexing control section 
123, when a service-data multiplexing request signal 

40 S1 5 is supplied from an external system controller 1 26, 
judges whether the amount of the service data D1X ac- 
cumulated in the buffer 1 25 is secured up to a minimum 
amount of data which can be transmitted. When the 
amount of data is secured, the section 123 transmits a 

45 buffer output control signal S13 for designating the 
amount of data equal to or less than the total amount of 
dummy data extracted by the invalid data extracting sec- 
tion 122, to the buffer 125 in which the service data D1X 
is accumulated. 

so [0063] Based on the buffer output control signal S1 3, 
the buffer 125 reads out the accumulated PES packets 
by the amount of data to be used as dummy data in ac- 
cordance with the first-in first-out method, and transmits 
the PES packets to the multiplexing section 127 as a 

55 service data stream D8X\ 

[0064] In this case, the multiplexing control section 
123 generates a new multiplexing schedule in order to 
newly multiplex the service data stream D8X' in addition 
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to the valid data streams D8A 1 and D8B' which are sup- 
plied from the invalid data extracting section 122, reads 
out the output data streams D8A and D8B from the buff- 
ers 103A and 103B in accordance with the schedule, in 
order to control the multiplexing of the multiplexing sec- s 
tion 127. 

[0065] Thus, the multiplexing section 127 generates 
TS packets having the fixed length in accordance with 
the valid data streams DBA' and D8B 1 supplied from the 
invalid data extracting section 1 22 and the service data io 
stream D8X' supplied from the buffer 125, multiplexes 
these TS packets, in order to generate a multiplexed 
stream (transport stream) D1 6 including the service da- 
ta (D1 X), and outputs the stream D16 to a transmission 
system (not illustrated) including a transmission anten- is 
na. 

[0066] Thus, the multiplexed stream D15 or D1 6 out- 
putted from the multiplexing section 127 is modulated 
by a modulator of the transmission system (not illustrat- 
ed) in a predetermined method, and the resultant trans- 20 
mission signal is transmitted toward a broadcasting sat- 
ellite (not illustrated) through the antenna. In this way, 
in the transmitter 120, the service data D1X is multi- 
plexed instead of extracted dummy data and is broad- 
casted in addition to television broadcast programs for 2s 
plural channels at the same time through the broadcast- 
ing satellite. 

[0067] In this connection, when there is no service da- 
ta D1X to be transmitted, the multiplexing control section 
123 multiplexes an invalid data stream corresponding 30 
to the amount of invalid data onto a valid data stream at 
the multiplexing section 127. 

[0068] In the case of the above structure, the data 
streams D8A and D8B outputted as the result of being 
compression-encoded by the encoders 1 01 A and 1 01 B 35 
respectively keep a certain rate by inserting dummy data 
(stuffing-byte data) between the header part and the val- 
id data part (after the header part) of a PES packet form- 
ing the stream D8A or D8B. In this case, the dummy 
data is inserted so as to be sprinkled in optional PES 40 
packets in accordance with the amount of encoded data 
generated at the encoders 101 A and 101 B. 
[0069] The dummy data is extracted by the invalid da- 
ta extracting section 122 of the multiplexer 121 and the 
valid data streams DBA' and D8B' are formed by closing 45 
the extracted portions. Therefore, the amount of data of 
the valid data streams D8A' and D8B' is reduced be- 
cause of extracting the dummy data. The amount of re- 
duced data is sequentially detected by the multiplexing 
control section 123 and accumulated by a counter so 
whenever dummy data is extracted. Therefore, the dum- 
my data sprinkled in the output data streams D8A and 
D8B is accumulated as a series of dead data whenever 
it is extracted by the data extracting section 122. 
[0070] The situation in which the amount of reduced ss 
data (the amount of dead data) reaches to a certain val- 
ue or more represents the situation in which the service 
data D1X that can be accumulated and transmitted at 



any timing can be multiplexed and transmitted to the 
multiplexed stream 16 every predetermined data unit. 
[0071] The multiplexing control section 1 23. receiving 
the service-data multiplexing request signal S15 from 
the system controller 126, decides whether the amount 
of reduced data due to dummy data extraction by the 
invalid data extracting section 122 reaches to a value 
for multiplexing the service data D1 X by predetermined 
amount. 

[0072] Since a negative result represents the situation 
in which the amount of dummy-data extracted by the 
invalid data extracting section 122 does not reach to a 
value capable of assigning the service data D1X, the 
multiplexing control section 123 waits for the amount of 
extracted dummy data to reach to a predetermined val- 
ue. 

[0073] On the other hand, when the amount of dummy 
data extracted by the invalid data extracting section 1 22 
reaches to a predetermined value, the multiplexing con- 
trol section 123 transmits the buffer output control signal 
S1 3 to the buffer 1 25 and thereby, reads out service data 
streams from the buffer 1 25 by a value equivalent to the 
amount of extracted dummy data in predetermined data 
unit, and supplies the data streams to the multiplexing 
section 127. 

[0074] Thus, in the multiplexing section 127, service 
data is multiplexed by the amount of data extracted by 
the invalid data extracting section 122, and the whole 
multiplexed stream D16 is transmitted while keeping a 
certain rate. 

[0075] According to the above structure, in the output 
data streams D8A and D8B respectively outputted from 
the encoders 1 01 A and 1 01 B, the invalid data extracting 
section 122, while extracting dummy data (stuffing-byte 
data) inserted every optional PES packet, counts the 
amount of extracted dummy data. Thereby, it is possible 
to multiplex and transmit significant service data instead 
of unnecessary dummy data and thus, multiplexing ef- 
ficiency can be improved because the multiplex data 
D16 does not include dummy data. 

(2) Second Embodiment 

[0076] Fig. 6 in which the same reference numerals 
are applied to parts corresponding to Fig. 2 shows a 
transmitter 130 used for a digital broadcasting system 
in a second embodiment. In Fig. 6, a multiplexing control 
section 1 32 is constituted so as to be able to control the 
number of codes generated by the encoders 101 A and 
1 01 B in addition to the structure and function of the mul- 
tiplexing control section 123 described in Fig. 2. 
[0077] In the second embodiment, when the multi- 
plexing request signal S15 is supplied to the multiplex- 
ing control section 132 from the external system con- 
troller 126, the service data D1X is multiplexed by the 
amount of extracted dummy data similarly to the afore- 
mentioned first embodiment. On the other hand, in the 
case where the multiplexing request signal S15 is not 
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supplied to the multiplexing control section 1 32 from the 
system controller 1 26 or in the case where the transmis- 
sion rate of the service data stream D8X is equal to or 
more than the amount of generated dummy data in spite 
of the presence of the multiplexing request, the multi- 
plexing control section 1 32 returns number-of-code con- 
trol signals S20A and S20B to the encoders 101 A and 
1 01 B based on the invalid-data-amount information sig- 
nal S1 0 obtained from the invalid data extracting section 
122. 

[0078] Fig. 7 in which the same reference numerals 
are applied to parts corresponding to Fig. 3 shows the 
encoder 101 A of the second embodiment. In Fig. 7, in 
addition to the structure and function of the control sec- 
tion 142A described in Fig. 3, a control section 242A is 
constituted so as to input a number-of-code control sig- 
nal S20A supplied from a multiplexer 131 and changes 
quantization step sizes of the quantizing section 1 4A, in 
order to control the number of codes to be generated. 
[0079] Moreover, the encoder 1 01 B also receives the 
number-of-code control signal S20Bfrom the multiplex- 
ing control section 1 32 and thereby changes quantiza- 
tion step sizes so as to control the number of codes to 
be generated. 

[0080] Therefore, in the case where the service data 
D1X is not multiplexed by the multiplexer 1 31 (Fig. 6), 
the multiplexing control section 132 transmits the 
number-of-code control signals S20A and S20B to the 
encoders 101 A and 101B and thereby, controls the 
number of codes so that the total number of codes gen- 
erated by the encoders 101 A and 101 B becomes equal 
to the number of codes obtained by adding a value 
equivalent to dummy data to the number of codes which 
have been already generated. 

[0081] However, in the case where the amount of 
dummy data extracted by the invalid data extracting sec- 
tion 1 22 is larger than the amount of service data to mul- 
tiplex the service data D1X by the multiplexer 131 (Fig. 

6) , the multiplexing control section 132 transmits the 
number-of-code control signals S20A and S20B to the 
encoders 101 A and 101 B and thereby, controls the 
number of codes so that the total number of codes gen- 
erated by the encoders 101 A and 1 01 B becomes equal 
to the number of codes obtained by adding the differ- 
ence between the amount of dummy data and the 
amount of service data to the number of codes which 
have been already generated. 

[0082] Therefore, in the quantizing section 14A (Fig. 

7) , more codes are generated and the amount of data 
accumulated in the buffer 141 A also increases. As a re- 
sult, the buffer 141 A can output the output data stream 
D8A having a predetermined rate only by significant da- 
ta (PES packet string comprising a header part and valid 
data) without adding dummy data (stuffing-byte data). 
Moreover, the second encoder 1 01 B can also output the 
output data stream D8B having a predetermined rate 
only by significant data similarly to the case of the en- 
coder 101 A. 



[0083] Thus, the above structure makes it possible to 
further improve transmission efficiency of significant da- 
ta because a certain rate can be kept without adding 
dummy data to the output data streams D8A and D8B 
s outputted from the encoders 1 01 A and 1 01 B. 

(3) Other Embodiment 

[0084] Note that, in the aforementioned embodiment, 
10 the invalid data extracting section 122 successively ex- 
tracts invalid data from the PES packets of the output 
data streams DBA and D8B, which are successively 
read out from the buffers 1 03A and 1 03B, in bytes (that 
is, every stuffing-byte data). However, the present in- 
is vention is not limited thereto but when all regions des- 
ignated by a header of a PES packet is dummy data, all 
the dummy data including the header can be used as 
invalid data. 

[0085] For example, as shown in Figs. BA and SB, in 
20 the case where invalid data parts ND/^ and ND^ are 
formed in, for example, the PES packets P A2 and P A4 
of the PES packet string P A1 , P^, P A3 , p Mt _ forming 
the output data stream DBA outputted from the encoder 
101 Aand invalid data parts ND B1 , ND B2 , and ND^ are 
2S formed in, for example, the PES packets P B1 , P B2: and 
P B3 ofthe PES packet string P B1 , P B2 , P B3 , P^,... form- 
ing the output data stream D8B outputted from the en- 
coder 101 B, the invalid data extracting section 122 (Fig. 
2 or 6) applies the same dummy-data detection and ex- 
30 traction as the case above described in Fig. 5 to the 
invalid data parts ND^, ND M , ND Bl , and ND B2 and 
thereby, extracts only these dummy data (invalid data 
part). 

[0086] On the other hand, in the PES packet P B3 in 
3S which dummy data (invalid data part D B3 ) is assigned to 
all the regions designated by the PES header, the invalid 
data extracting section 122 extracts not only the invalid 
data part ND^, which is constituted with the dummy da- 
ta, but also the header part H B3 of the PES packet as 
40 invalid data parts. In this case, in the header part H^, 
the type of data (normally indicating the type of picture 
data or the like) assigned to the region (invalid data part 
ND B3 of the PES packet P B3 ) designated by the header 
part Hb3 is rewritten by the encoder 101 B so as to indi- 
es cate dummy data (stuffing-byte data) and therefore, the 
invalid data extracting section 122 can detect that dum- 
my data is assigned to all the regions designated by the 
header H^ of the PES packet P B3 by reading out the 
header part H B3 . 
50 [0087] Thus, in the case where dummy data is as- 
signed to all the regions designated by the header part 
H B3 , it is possible to handle the header part H B3 as 
invalid data. In this case, the invalid data extracting sec- 
tion 122, as shown in Fig. 8F, extracts the header part 
55 H B3 as invalid data and thereby, transmits the amount 
of reduced data including the header part H B3 to the mul- 
tiplexing control section 1 23 (or 1 32) as the invalid-data- 
amount information S10. Thus, the multiplexing control 
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section 123 (or 132) recognizes the amount of data in- 
cluding the header part H B3 as the amount of invalid da- 
ta. Thereby, it is possible to further improve multiplexing 
efficiency, such as the case of raising a rate for multi- 
plexing the service data D1 X and the case of increasing 
the number of codes to be generated by the encoding 
sections 101 A and 101 B. 

[0088] In this connection, Figs. 8G and 8H show the 
valid data streams D8A' and D8B' after invalid data is 
extracted. 

[0089] Moreover, in the above embodiments, the en- 
coders 101 A and 101 Band the buffers 103Aand 103B 
are provided correspondingly with two channels so as 
to multiplex the two output data streams D8A and D8B. 
However, the present invention is not limited thereto but 
three or more channels can be used. In this case, en- 
coders and buffers equal to the number of channels can 
be provided to multiplex output data streams equal to 
the number of channels similarly to the aforementioned 
method. 

[0090] Furthermore, in the above embodiments, the 
output data streams D8A and D8B outputted from the 
encoders 101 A and 101B are respectively handled as 
data streams comprising a PES packet string. However, 
the present invention is not limited thereto but it is also 
possible to apply the present invention to a case of out- 
putting the data streams in another data unit. 
[0091] Furthermore, in the aforementioned embodi- 
ments, the present invention is applied to the transmit- 
ters 120 and 130 of a digital broadcasting system ac- 
cording to the PEG2 method. However, the present in- 
vention is not limited thereto but it can be applied to other 
various transmitters adopting other compression-en- 
coding methods. 

[0092] Furthermore, in the aforementioned embodi- 
ments, the service data D1X for one channel is multi- 
plexed instead of invalid data. However, the present in- 
vention is not limited thereto but the service data for two 
channels or more can be multiplexed. In this case, a plu- 
rality of channels comprising an encoder 124X and the 
buffer 125 is provided. 

[0093] Furthermore, in the aforementioned embodi- 
ments, the quantization step sizes of the encoders 1 01 A 
and 101B are controlled by the number-of-code control 
signals S20A and S20B outputted from the multiplexing 
control section 1 23 (or 1 32). However, the present in- 
vention is not limited thereto but a data rate to be out- 
putted from the output stage of the buffer section 141 A 
(1 41 B) of the encoders 1 01 A and 1 01 B can be changed. 
[0094] According to the present invention as de- 
scribed above, multiplexing efficiency of valid data can 
be improved by detecting and extracting invalid data as- 
signed to an inputted data stream and multiplexing and 
outputting the data stream from which the invalid data 
is extracted. 

[0095] While there has been described in connection 
with the preferred embodiments of the invention, it will 
be obvious to those skilled in the art that various chang- 
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es and modifications may be aimed, therefore, to cover 
in the appended claims all such changes and modifica- 
tions as fall within the true scope of the invention. 



Claims 

1. A data multiplexer for multiplexing and outputting 
data streams of plural channels, comprising: 

invalid data extracting means for detecting and 
extracting invalid data assigned to said data 
streams; and 

multiplexing means for multiplexing and output- 
ting the data streams from which said invalid 
data is extracted. 

2. The data multiplexer according to claim 1 , wherein: 

said data stream is composed of plural packets, 
wherein said packet has a data part and a head- 
er part 

including information on invalid data which said 
data part has; and 

said invalid data extracting means extracts said 
invalid data based on the information of said 
header part. 

3. The data multiplexer according to claim 2, wherein 

said invalid data extracting means, in the case 
where all data in said data part of said packet are 
invalid data, extracts said data part and said header 
part of said data part. 



35 4. 



40 



45 



SO 



55 6 . 



The data multiplexer according to claim 1 , wherein: 

said data stream has time information; and 
said time information is renewed when said 
invalid data extracting means detects and ex- 
tracts invalid data assigned to said data stream. 

The data multiplexer according to claim 1 , compris- 
ing; 

invalid data amount detecting means for detect- 
ing the amount of said extracted invalid data; 
and 

said multiplexing means for multiplexing a pre- 
determined data stream onto the data stream 
from which said invalid data is extracted in ac- 
cordance with the amount of invalid data de- 
tected by said invalid data amount detecting 
means. * 

The data multiplexer according to claim 5, compris- 
ing: 

input means for inputting outside data from the 
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outside; and 

encoding means for encoding said outside da- 
ta, wherein 

said predetermined data stream is data gener- 
ated by encoding said outside data by said en- 5 
coding means. 

7. The data multiplexer according to claim 1 , compris- 
ing; 

10 

invalid data amount detecting means for detect- 
ing said amount of extracted invalid data; and 
outputting means for outputting a control signal 
for controlling the amount of produced valid da- 
ta included in said data stream in accordance is 
with the amount of invalid data detected by said 
invalid data amount detecting means to a data 
encoder serving as a source for outputting said 
data stream, wherein 

when the amount of invalid data included in said 20 
data stream to be inputted increases, the 
amount of valid data to be produced in said data 
encoder is increased. 

8. The data multiplexing method of multiplexing and *s 
outputting data streams of plural channels, com- 
prising: 

an invalid data extracting process of detecting 
and extracting invalid data assigned to said da- 30 
ta stream; and 

an multiplexing process of multiplexing and out- 
putting a data stream from which said invalid 
data is extracted. 

35 

9. The data multiplexing method according to claim 8, 
wherein 

said data stream is composed of plural packets, 
wherein said packet has a data part and a head- 40 
er part 

including information on invalid data which said 
data part has; and 

said invalid data extracting process extracts 
said invalid data based on the information of 45 
said header part. 

10. The data multiplexing method according to claim 9, 
wherein 

said invalid data extracting process, in the so 
case where all data in said data part of said packet 
are invalid data, extracts said data part and said 
header part of said data part. 

1 1 . The data multiplexing method according to claim 8, ss 
wherein: 

said data stream has time information, and said 



data multiplexing method comprising 
a renewing process of renewing said time infor- 
mation in the case of extracting invalid data as- 
signed to said data stream in said invalid data 
extracting process. 

12. The data multiplexing method according to claim 8, 
comprising: 

invalid data amount detecting process of de- 
tecting said amount of said extracted invalid da- 
ta; and 

said multiplexing process of multiplexing a pre- 
determined data stream onto the data stream 
from which said invalid data extracted in ac- 
cordance with the amount o1 invalid data de- 
tected by said invalid data amount detecting 
process. 

13. The data multiplexing method according to claim 
12, comprising: 

an inputting process of inputting outside data 
from the outside; and 

a encoding process of encoding said outside 
data, wherein 

said predetermined data stream is data gener- 
ated by encoding said outside data by said en- 
coding process. 

14. The data multiplexing method according to claim 8, 
comprising: 

an invalid data amount detecting process of de- 
tecting said amount of extracted invalid data; 
and 

an outputting process of outputting a control 
signal for controlling the amount of produced 
valid data included in said data stream in ac- 
cordance with the amount of invalid data de- 
tected by said invalid data amount detecting 
process to a data encoder serving as a source 
for outputting said data stream, wherein 
when the amount of invalid data included in said 
data stream to be inputted increases, the 
amount of valid data to be produced in said data 
encoder is increased. 
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